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Electronic portfolios have become increasingly popular. The value of a portfolio, though, depends on 
how, when, and why students create, submit, and have their portfolios evaluated. In the following 
paper, we describe how we redesigned a program's assessment and evaluation plan around the use of 
electronic portfolios and a system of gates focusing on the larger assessment/evaluation framework 
first and the technology second. 


Institutions of Higher Education find themselves in 
an age of accountability (Alexander, 2000; Gansemer- 
Topf & Schuh, 2006; Welsh & Metcalf, 2003). 
Stakeholders such as accreditors, policymakers, alumni, 
parents, and students are demanding proof from 
institutions of higher education of quality teaching and 
student learning like never before (Millett, Payne, 
Dwyer, Stickler, & Alexiou, 2008). Increased focus has 
especially been placed on institutions that offer courses 
and programs online (Gabriel, 2010; Means, Toyama, 
Murphy, Bakia, & Jones, 2010; Martinez, Liu, Watson, 
& Bichelmeyer, 2006; Rovai, 2003) and recently there 
is even a push for institutions to better prepare 
graduates for gainful employment (Hamilton, 2010). 
The focus on accountability appears to be here to stay 
and the burden of proof lies on the shoulders of 
institutions of higher education to provide evidence of 
high quality teaching and learning. As a result, many 
teacher education programs have turned to portfolios to 
address these new accountability mandates from 
accreditors and other stakeholders (Fiedler, Mullen, & 
Finnegan, 2009). 

Portfolios have become a popular form of 
assessment across all fields and levels of education 
(Brickley, Schwartz, & Suen, 2000; Chen & Black, 
2010) but especially in our field—the field of teacher 
education (Maher & Gerbic, 2009; Strudler & Wetzel, 
2005). The use of portfolios (or specifically portfolio 
assessment) in education began in the late 1980s (Barrett, 
2007). However, the use of portfolios did not come into 
widespread practice until the late 1990s (Barton & 
Collins, 1993; Strudler & Wetzel, 2005; Wade & 
Yarbrough, 1996). Since that time, electronic portfolios 
in particular have become increasingly popular (Barrett, 
2002; Penny & Kinslow, 2006; Strudler & Wetzel, 2005; 
Williams, Wetzel, & Wilhelm, 2004). In fact, electronic 
portfolios, which we will refer to as ePortfolios 
throughout this paper, have been described as “higher 
education’s new 'got to have if tool” (Cohn & Hibbitts, 
2004, p. 7). Some have even gone so far to suggest that 
ePortfolios have the potential to change higher education 
significantly (Love, McKean, & Gathercoal, 2004; 


Treuer & Jenson, 2003). Not surprisingly, colleges and 
universities are rushing to find ways to use this “got to 
have it” tool—especially in this new age of 
accountability. Overall, though, colleges and universities 
are having mixed results with implementing ePortfolios 
into their programs (as suggested in Fielder, Mullen, & 
Finnegan, 2009; Gathercoal, Love, Bryde, & McKean, 
2002; Love et ah, 2004). 

Despite the lack of empirical evidence supporting 
the benefits of ePortfolios (Evans & Powell, 2007; 
Hartmann & Calandra, 2007; Reardon & Hartley, 

2007) , educators have identified a number of benefits of 
ePortfolios. These benefits include experience using 
and learning about computer applications (Lin, 2008; 
Milman & Kilbane, 2005; Wall, Higgins, Miller, & 
Packard, 2006), easy access and updates to portfolios 
(Jun, Anthony, Achrazoglou, & Coghill-Behrends, 
2007; Strudler & Wetzel, 2008), promoting reflection 
(Lin, 2008; Strudler & Wetzel, 2008), supporting 
formative assessment (Wall et ah, 2006), tracking 
student learning and performance for accreditation and 
program evaluation purposes (Strudler & Wetzel, 

2008) , improving communication between faculty and 
students (Strudler & Wetzel, 2008), landing a job after 
graduation (Strudler & Wetzel, 2005; Ward & Moser, 
2008; Wetzel & Strudler, 2006), and fostering lifelong 
learning (Heinrich, Bhattacharya, & Rayudu, 2007) to 
name a few. 

Therefore, regardless of the “fad” factor, there are 
some sound reasons that programs, colleges, and 
universities express interest in, if not completely 
implement, an ePortfolio system. However, 
implementing an ePortfolio system can be challenging 
(Gathercoal et ah, 2002; Love et ah, 2004). And in 
fact, while some like Garthercoal et al. (2002) suggest 
implementing an ePortfolio system is easier when an 
academic unit already uses a paper portfolio, we 
contend that it can be harder to transition from a 
paper-based portfolio to an ePortfolio system than it is 
to start from nothing because of the assumptions and 
ways of doing things that faculty and staff might carry 
with them. 
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When faced with the task of converting a 
traditional summative paper-based portfolio to an 
ePortfolio, faculty and administrators often simply 
create an electronic version of the old paper-based 
portfolio (see Treuer & Jenson, 2003). We understand 
why faculty and staff might choose to do this; rather 
than disrupt a system that “works,” faculty and staff opt 
to change as little as possible—in part to help maintain 
the status quo. However, simply creating an electronic 
version of a paper-based portfolio does not necessarily 
improve much (as alluded to by Treuer & Jenson, 
2003). That is, simply making something “electronic” 
by putting it on the Web does not necessarily make it 
better (Bauerlein, 2008; Keen, 2008; Oppenheimer, 
1997); in fact, we posit at times it can even make things 
worse by adding additional obstacles. Further, a 
portfolio—whether paper-based or electronic—is only 
as good as the larger assessment and program 
evaluation framework it is situated within (Fielder, 
Mullen, & Finnegan, 2009). We contend that much of 
the value of a portfolio (whether electronic or not) 
depends on how, when, and why students create, 
submit, and have their portfolios evaluated. 

In the following paper, we share our experience 
redesigning a program’s assessment and evaluation plan 
around the use of ePortfolios—using ePortfolios both as 
a means toward ensuring student learning throughout a 
program of study (as opposed to only at the end) 
through the use of regular reviews as well as a means 
toward large-scale program evaluation. As such, this is 
more of a story about how and why one program turned 
to the use of ePortfolios to improve and document 
teaching and learning, than it is a specific blueprint or 
empirical study on how to implement ePortfolios across 
all contexts. However, we believe that administrators, 
faculty, and staff across different disciplines will find 
our story about redesigning our program around 
ePortfolios compelling and useful. 

Background 

Some background information is needed to better 
understand our unique context as well as the decisions 
we made along the way. We, the three authors, were 
working at a Teacher Education Department at a private 
Catholic university in a western state in the United 
States. For the purpose of this paper, we will refer to 
this university as Catholic Western University (CWU). 
CWU though is not a traditional university. All of the 
programs in the School of Education and Counseling at 
CWU are accelerated programs offered year round. The 
academic year consists of six 8-week terms. In addition 
to the accelerated nature of the program, the teacher 
licensure students in the program are also dispersed 
across three western states—taking courses either 
online, through independent study, or face-to-face (and 


Improving Assessment and Program Evaluation 62 


often a combination of the three). The average student 
in the program is 37 years old. 

Our program, like many teacher education 
programs, required our teacher licensure students (i.e., 
students preparing to get credentialed as K-12 teachers) 
to complete a paper-based portfolio at the end of their 
program to demonstrate what they “know and are able 
to do.” 

For years, our students would compile their paper- 
based portfolio and have it evaluated during the last 
semester of their program. While we the faculty at 
CWU preferred a portfolio method of assessing student 
learning (as opposed to simply relying on course grades 
or some type of exit exam), it became apparent over the 
years that our traditional summative paper-based 
portfolio system was not working as well as it could 
have been or even as it was intended. 

Shortcomings of our Portfolio Process 

It is easy in hindsight to identify why our portfolio 
method of assessing student learning was not working 
as well as it could have been. The portfolio became 
more of a box on a checklist that needed to be checked 
off than a meaningful or effective way to assess student 
learning (for students as well as faculty). This is in part 
because of the lack of purpose (i.e., faculty were 
unclear whether the portfolio was meant to serve as a 
reflective portfolio, a summative assessment portfolio, 
or a showcase portfolio), lack of structure (i.e., students 
could wait until their last semester before they began to 
create their portfolio), lack of emphasis (i.e., because 
students could and often did wait until their last 
semester to compile their portfolio, often it was rushed 
and simply not given enough time or focus), lack of 
specificity (e.g., students could choose what they 
included in the portfolio), lack of differentiation (e.g., 
undergraduate and graduate, despite the focus of their 
program essentially completed the “same” portfolio), 
and lack of consistency/reliability (e.g., expectations of 
faculty varied greatly about what a “proficient” artifact 
looked like). As a result, the portfolio system was not 
adequately demonstrating student learning or mastery 
of course content and was not being assessed 
consistently and adequately at the end of the students’ 
programs. It took an outside entity, though, to help us 
recognize that our portfolio system was not working 
effectively. This is not as strange as it sounds, and some 
might even argue that this is the purpose of 
accreditation visits because many other institutions 
begin implementing e-portfolios as a result of 
accreditation (Love et al., 2004). 

Around 2003, our program was confronted with the 
reality that the majority of the teacher education 
programs in the state were accredited by one of two 
teacher education accrediting bodies—the National 
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Council for Accreditation of Teacher Education 
(NCATE) or the Teacher Education Accreditation 
Council (TEAC). Over time, due to mounting political 
pressures from the State Department of Education 
coupled with a desire to remain competitive with other 
programs in the state and ultimately a desire to improve 
the quality of our program—we chose to seek national 
accreditation through TEAC. 

TEAC Accreditation and Self-Study as a Catalyst 
for Change 

We chose to seek accreditation through TEAC, like 
a growing number of other universities (Bollag, 2006), 
largely because of their evidence and claim-driven 
process (as opposed to NCATE’s standards-based 
process) (Murray, 2000). As an accelerated non- 
traditional adult program serving three different states, 
our program was anything but “standard.” TEAC’s 
claim-driven and evidence-based model allows 
institutions to provide evidence for claims it makes 
about what their students know and can do rather than 
meet nationwide standards that might not apply to their 
programs or population. 

After meeting TEAC’s eligibility requirements, we 
conducted an internal audit and began preparing our 
Inquiry Brief. An Inquiry Brief is a self-study document 
in which a program provides evidence that it is 
producing “graduates who are competent, caring, and 
qualified educators, and that the program has the 
capacity to offer quality” (TEAC, 2009, para 2). 
Everyone involved in our program learned a great deal 
during this self-study process. Through this process, we 
realized that our assessment process—which relied 
heavily on evaluating students’ final paper-based 
summative portfolios—was not providing reliable and 
valid evidence of student growth, mastery of state 
standards, or adequate data for larger program 
evaluation. Even before being audited by TEAC, it 
became clear that we did not have enough reliable and 
valid data to support the claims we made about student 
learning in our program. It was not that we lacked data; 
rather, we lacked the appropriate type of data. For 
instance, student grades alone are not reliable or valid 
enough to support claims about student learning. 
Moreover, while our administration had instituted an 
electronic data collection system—based in part on the 
paper-based portfolio—it resulted in large amounts of 
unreliable, invalid, and ultimately unused data. Not 
surprisingly, after the audit, TEAC specifically pointed 
out weaknesses in “Evidence of valid assessment” and 
“Program decisions and planning based on evidence.” 

Through the TEAC audit, we realized (some of us 
faster than others) that the problem was not the lack of 
data but rather that the wrong type of data was being 
collected at the wrong times (and largely without a 
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standard means of gathering and interpreting said data). 
The data we were collecting was not standardized and it 
provided little evidence of whether or not instructors 
were providing similar (and quality) instruction based 
upon specific criteria. TEAC helped illustrate this by 
getting a group of us in a room together and asking us 
to all assess the same exact thing. The results, as you 
might imagine, were not consistent. In short, we had no 
valid means of evaluating the quality of varied (and 
widely-dispersed) affiliate faculty (i.e., part-time 
adjunct faculty) and their courses other than grades and 
student satisfaction surveys (which research suggests is 
not an adequate measure of teaching quality). What we 
wanted and needed was a means of tracking student 
learning as well as assessing the standardized quality of 
instruction across instructors and courses. 

It became increasingly clear that to meet TEAC’s 
requirements for accreditation, a new assessment and 
evaluation system was required. That is, rather than 
simply creating an electronic version of a paper-based 
system or adding electronic components to a paper- 
based assessment system, a new system needed to be 
developed. This was not an easy decision to come to 
because we all knew how much work a complete 
overhaul of our current assessment system would take. 
But in the end, we believed the hard work would pay 
off and result in more meaningful assessment and 
evaluation. 

In summary, information obtained from our self- 
study and accreditation visit suggested that we did not 
have the type of evidence of student learning that we 
thought we had and needed for national accreditation. 
We needed a new means for obtaining evidence of 
student learning and growth over time and a means for 
ensuring standardization across affiliate faculty and 
courses. 

Program Changes and Assessments 

Due to the results of our self-study and 
accreditation visit, we had to make some important 
decisions about what we could and should change in the 
program. While we were comfortable with eliminating 
the old assessment system, we were not interested in 
making any more changes than needed. Given this, we 
decided to make some important changes that are 
addressed in the following pages. 

Proficiencies 

When confronted with the need to start over and 
build a reliable and valid assessment and evaluation 
system, following good practice, we began by asking, 
“What do students with a degree in education need to 
know and to be able to do?” Typically teacher 
educators would turn to state or national standards to 
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Table 1 

Universal Proficiencies 

Proficiency 

Description 

Critical Thinking: 

The student will be able to gather information from observation, experience, reasoning, 
and/or communication, analyze that information, generate alternatives, solve problems, 
and evaluate the process and solution. Critical thinking is based on intellectual values 
that go beyond subject matter to include clarity, accuracy, precision, evidence, fairness, 
and multiple perspectives. 

Learning Theory: 

The student will have knowledge of the complex process of how people learn and will 
be able to apply a variety of learning theories in an educational setting. 

Professionalism: 

The student will have the ability to represent the teaching profession effectively by 
demonstrating the following characteristics: commitment to learning, adherence to 
ethical standards, respect for diversity, effective communication skills, effective 
interpersonal skills, and accountably for actions. 

Assessment: 

The student will understand and apply the principles of measurement, analysis, and 
decision making about what students know and are able to do. 

Instruction: 

The student will understand and use research-based strategies and techniques to 
facilitate student learning and to differentiate instruction based upon individual 
students’ needs. 

Technology: 

The student will demonstrate understanding and appropriate applications of technology 
as they relate to effective instruction and to specific endorsement areas. 

Values: 

The student will demonstrate an understanding of democracy, ethics, moral integrity, 
multiculturalism, social justice, and the concept service learning. 

Communication: 

The student will be able to communicate effectively through speaking, writing, 
listening, and observing. Students will understand effective ways of talking with 
students and demonstrate appropriate communication skills to their students. 

Discipline Knowledge: 

The student will demonstrate proficiency in the specific content area(s) of their 
program, licensure and/or endorsement area. 


answer a question like this. However, as a 
nontraditional teacher education program with 
students spanning three different states (and therefore 
three different state departments of education), three 
different state or national standards actually made this 
a very complicated question to answer. To make 
matters worse, in addition to three different sets of 
state standards, we also had years of additional 
outcomes being “added” on to our three sets of state 
standards, totaling over 90 different outcomes in all. 
After some careful reflection and lively discussions, 
we came to the conclusion that by trying to assess 
everything, we were actually not able to truly assess 
anything meaningfully. In hindsight, this is not 
surprising; this is the basic quantity versus quality 
dilemma faculty regularly face. 

Rather than continue striving to address 90 
different outcomes, we decided to synthesize the 


different outcomes into a short and hopefully 
meaningful set of outcomes we decided to call 
“Proficiencies.” We created a proficiencies table 
(called the “cross walk”) to illustrate to each 
stakeholder (e.g., the three different state 
departments of education) how each standard and 
outcome is addressed. The proficiencies consist of 
both a core set of proficiencies that all students 
getting a degree in education must meet (which are 
called “Universal Proficiencies” and listed in Table 
1) as well as a list of discipline specific proficiencies 
that only pertain to certain fields of study (e.g., 
students seeking a license as a special education 
teacher have a different set of discipline specific 
proficiencies than students seeking a license as a 
math teacher). Discipline knowledge proficiencies 
were created for each of the different teacher 
licensure areas. 
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After the Universal Proficiencies and Discipline 
Proficiencies were created for each licensure program, 
we reviewed the curriculum to identify what courses 
and assignments best addressed the proficiencies. That 
is, we mapped the proficiencies to the curriculum. 
While the majority of the proficiencies were easily 
mapped to courses and assignments, there were some 
instances where we had to update the curriculum to 
address certain proficiencies. 

Ultimately we hoped that by identifying fewer 
program outcomes and mapping those directly to 
specific courses and assignments, faculty and students 
should be better able to focus on addressing each 
proficiency in more depth. 

The Gate System 

Perhaps the biggest change we made was the 
creation of a Gate System—that is stages in each 
program where students must successfully pass a 
portfolio review to continue on in the program. A Gate 
System though would not have been realistically 
possible without utilizing an ePortfolio, given the 
accelerated nature of the program and the geographic 
dispersion of the faculty and students. The following 
pages specifically focus on describing the Gate System 
we developed and the role that on-going assessment, 
gates, a portfolio coordinator, and gatekeepers play in 
making this system work. 

On-going assessment. One of the problems with 
waiting until the end of a student’s program to assess 
what they have learned in a summative portfolio is that 
it is often too late to do much about it. For instance, 
after students have completed all of their coursework in 
a given program with passing grades, it becomes very 
difficult in our experience to hold students accountable 
for any gaps in their knowledge. On-going assessment 
provides faculty and administrators opportunities to 
formatively assess students’ knowledge throughout 
their program and make adjustments as necessary. 
Therefore, we decided to change the portfolio 
development and review process to include on-going 
assessment. Rather than creating a cumulative and 
largely post-hoc portfolio in the final course of the 
program, we decided to require our students to begin 
contributing to their portfolio during their first courses 
and build on their portfolio after each course they 
complete. 

We designed the system so that students would 
complete the assignments (which we refer to as 
“required artifacts”) aligned with the proficiencies in 
each course and then include them in their portfolio. 
Further, each proficiency was intentionally designed to 
be addressed and assessed at least twice in each 
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student’s program (thus providing a means of 
demonstrating growth and learning over time). 

Rubrics were created for each of the required 
artifacts and included in the syllabus for each course. 
The entire process was designed to be as transparent as 
possible. Faculty (at the course level) were then trained 
on how to use the rubrics to help ensure consistency. 
The system was set up so that as the required artifacts 
and rubrics are updated, faculty would receive 
additional training on how to use the rubrics. In 
addition, the assessment system was set up so that each 
year the accumulated data from the completed rubrics 
could be evaluated to ensure the reliability of the 
rubrics as measurement tools (to check for variance in 
inter-rater reliability). 

This standardization and consistency was setup so 
that students could also improve upon their artifacts 
(after receiving feedback from their instructors) before 
adding it as an artifact to their ePortfolio. This provides 
students the ability to improve their artifacts (and 
evidence of learning) prior to submitting the artifacts 
for review in their portfolios. 

Gates. A key component to making on-going 
assessment meaningful in our system is through the use 
of a gate system—that is, a series of stages or gates 
where students must successfully pass a portfolio 
review in order to continue on in their program. Gates 
in a portfolio system serve a few purposes. First, they 
provide students with a clear incentive to begin creating 
their portfolio in their very first course. Second, they 
provide students an opportunity to have an independent 
reviewer (i.e., someone other than the instructor of their 
course) review their work for evidence of learning. 
Third, the gates provide faculty, students, and staff an 
opportunity to ensure that every student has 
demonstrated that he or she has learned the required 
skills and dispositions to proceed in the program. This 
becomes really important in programs like ours where 
students have to complete a capstone experience in a 
professional setting. In our field, students have to 
successfully complete a student teaching experience in 
a K-12 classroom in order to graduate. Nothing is worse 
than placing a student teacher in a classroom when they 
are not prepared or ready to be there. The gate system 
was intentionally designed to serve each of these 
purposes. 

While there are a number of ways a gate system 
can be used, we designed our system so that each 
program has two-to-three gates throughout the program 
(see Figure 1). When students are ready to submit their 
portfolios for review, all they need to do is to send a 
request for their portfolio to be reviewed to a person 
designated as the Portfolio Coordinator—a staff person 
in charge of assigning reviewers to each portfolio. 
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Figure 2 

Detailed View of a Gate System 
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Portfolio coordinator. Faculty already feel 
overburdened in this age of accountability. A Portfolio 
Coordinator is a necessary component to a successful 
gate system—especially in an accelerated program like 
ours where students can be submitting their portfolios 
for review every eight weeks. Our system was set up so 
that once the Portfolio Coordinator receives a request 
for a portfolio to be reviewed, he or she then assigns an 
independent reviewer (a faculty member called a 
“Gatekeeper” which is described in the following 
section) to review and evaluate the portfolio and 
follows up to ensure that the review is completed (see 
Figure 2). 

Gatekeepers. One of the problems we found with 
our previous paper-based summative portfolio process 
was that faculty were often biased when it came time to 
evaluate students’ portfolios due to prior relationships 
they had with the students. Therefore a key component 


we designed into our assessment and evaluation system 
was the use of Gatekeepers. Gatekeepers are current 
faculty members (primarily part-time affiliate faculty 
members) who have expertise in specific areas related 
to a specific gate as well as a willingness to serve in the 
role of a portfolio reviewer (i.e., someone responsible 
for assessing student portfolios). 

Gatekeepers are paid to assess student portfolios 
for pre-determined content using standardized rubrics in 
which they have been trained to use. They use a 
standardized rubric to assess students’ individual 
artifacts; the rubric helps them calculate a cumulative 
score for each student’s portfolio which is used to 
determine whether the student has met a given standard 
level of knowledge and performance (appropriate to the 
student’s stage in the program) and can continue on in 
the program. Specific feedback on the strengths and 
weaknesses of each artifact and the quality of the 
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Figure 3 

A Gate System Remediation Plan 





portfolio (as it stands at each gate) are provided to each 
student after review of their portfolio at each gate. 

The rubrics were set up to be completed 
electronically for each portfolio so that the results could 
be stored in a central database that can later be mined 
for reports for accreditation visits as well as faculty, 
course, and program evaluation purposes. This process 
also enables Gatekeepers to be periodically assessed for 
consistency of reviews—which includes comparing 
their ratings and feedback to other gatekeepers. 

Remediation. We recognize though that 
occasionally students might not pass a given gate for a 
variety of reasons. If upon first review a student’s 
portfolio does not meet a minimum numerical 
aggregate score based on the weighted rubric, the 
student is notified of the failing score, with specific 
feedback (a copy of the evaluation rubric with evaluator 
comments), and provided an opportunity to correct 
errors, weaknesses, etc. (which is referred to as the 
remediation plan) and resubmit the portfolio for a 
second review (see Figure 3). If, however, a student’s 
portfolio fails a second review by a second gatekeeper, 
the student can be removed from the program. Students 
can also be placed on a remediation plan as the result of 
a negative professional disposition (i.e., an evaluation 
completed by either a faculty member or someone in 
the field about a student’s professional demeanor). 
Students reaching this point—either because their 
portfolio failed to pass the gate two times in a row or 
because of negative dispositions—may appeal a 
decision to be removed from the program. 


ePortfolios 

As mentioned earlier, none of this would be 
realistically possible given the accelerated and 
geographically separate nature of our programs and 
students in a paper-based portfolio. By requiring 
students to use an ePortfolio, students are able to do the 
following: 

• begin working on their portfolio at the start of 
their program; 

• collect artifacts (and at times iterations of 
artifacts ) throughout their program; 

• have a record of faculty assessments of their 
artifacts; and 

• quickly submit their portfolio for review at any 
time throughout their program and expect a 
quick response. 

Finally, while the ePortfolio is primarily a 
combination of a development and assessment 
portfolio, students can quickly create a showcase 
portfolio using artifacts of their choice later when it is 
time to look for a job. 

Given these benefits, after reviewing a number of 
different tools, we selected to use iWebfolio as the 
ePortfolio platform for our programs. However, we 
believe our entire system in many ways is not 
dependent on any single ePortfolio platform and instead 
can be used with a variety of different tools. We 
require our students to purchase an iWebfolio account 
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before their first course (see http://www.iwebfolio 
.com). We identified a preferred first course for each 
program. In this first course, students are oriented to the 
portfolio review process and iWebfolio. We also 
provide workshops and other support materials for 
students year round. 

Program, Faculty, and Course Evaluation 

The gate system and the larger ePortfolio 
assessment and evaluation system was designed to 
enable our administrators the ability to look for trends 
when students do very well on a certain artifact and 
similarly when students do not perform well on a 
certain artifact. Not only can this data provide a means 
of assessing student learning and performance, it also 
can provide a means for gauging faculty and course 
effectiveness (including such things as grade inflation). 

Research has shown that student achievement is 
directly related to teacher quality (Darling-Hammond, 
2000). In the past, faculty members have been assessed 
through a college-wide initial faculty assessment 
(which, because of its brevity, its failure to 
authentically represent what happens in a real 
classroom teaching environment, and because it is a 
pre-assessment of teaching ability has proven to be 
inadequate for gauging faculty effectiveness) and 
through end of course evaluations. But through the 
systematic collection of student assessment data, 
faculty—and specifically teacher quality—can now be 
evaluated in a way like never before. 

Data on student performance can be cross- 
referenced against instructors for the course from which 
the artifacts were created. This means that, if we begin 
to notice poor student performance on a given artifact, 
we can determine if said artifacts are originating in 
courses taught by specific instructors. Such a finding 
could suggest that the instructor is failing to teach the 
content of the artifact sufficiently or that a change in 
instructional practices is needed. If, however, 
administrators discover that students across instructors 
are performing poorly on a given artifact, they can 
assess the appropriateness of the chosen artifact to the 
course content, whether or not specific content is being 
addressed in the course as a whole, whether or not 
instructors are following the module (i.e., the master 
course syllabus used by all instructors of a given 
course), as well as whether or not the artifact chosen for 
said course or course format needs to be revised. 

At the same time, our system was setup to enable 
us to annually analyze the data collected on how each 
gatekeeper rates each artifact individually at each gate. 
By disaggregating this data, the assessment system can 
provide feedback on inter-rater reliability among 
gatekeepers and help us determine if changes in 
gatekeepers, their training, or the rubrics is needed. 
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This type of data coupled with course grades and 
end of course evaluations will enable us in the coming 
years to make data-driven decisions about what 
improvements need to be made in our programs and 
courses. 

Conclusion and Future Trends 

Our assessment and evaluation system—which is 
built upon the concept of on-going assessment, gates, 
gatekeepers, and the electronic storage and 
dissemination of artifacts in an ePortfolio—is still in its 
infancy. In fact, this new system was officially started 
about a year ago. Therefore, in many ways it is too soon 
to assess its effectiveness. However, initial results 
suggest that overall it is working just as designed. 
Students are working on their portfolios throughout 
their programs—beginning with their first course. They 
are also getting feedback at each gate about how they 
are performing and any gaps in their knowledge (based 
on the artifacts submitted). Over time though, it is 
assumed that courses will need to be updated, artifacts 
and rubrics improved, and the system management 
processes tweaked. 

Future trends for the improvement of the system 
include changing, adding to, and eliminating some of 
the present artifact assignments that are required to be 
in each ePortfolio. That is, while the overall number 
one purpose of this assessment and evaluation system is 
to more effectively assess student learning at different 
stages (i.e., gates) of each student’s program and 
ultimately to prepare the best teachers possible, we 
want to ensure that the workload involved in the day-to- 
day operation of this system remains realistic and 
manageable. Therefore, just as components can be 
added and adapted as needed, over time certain things 
might be dropped if found unnecessary. 
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